Cognitive Coupling of Animation and User Action in Interactive Digital Artworks

We argue, from a perspective integrating cognitive science and the arts, that interactive animated visual graphics, as embodied images whose understanding relies on users’ perceptual and motor apparatuses, connect both mental and material notions of images. Drawing upon cognitive science theories of conceptual blending and material anchors, recent neuroscience results regarding mirror neurons, and phenomenological approaches to human-computer interaction, we bridge the gap between visual perception and bodily interaction in digital environments – calling this process as “material-based imagination” (in contrast to the general notion of imagination as purely a mental activity). Animated images trigger a re!ective process in which material-based imaginative construction and elaboration can take place. When this process, enabled by today’s real-time control and rendering technologies, becomes instantaneous and continuous, it mobilizes a motor-sensory feedback loop. This type of user experience constitutes embodiment of meaning and intention through interaction with digital media artifacts. This kind of embodied understanding is pervasive in today’s digitally mediated environments. Through analyses of digital artwork, we show the important role of imaginative blends of concepts in making multiple levels of meaning through embodied expressiveness with motion-based motor input. The implications of these analyses collectively form a step toward an embodied cognition approach to animation phenomena and toward recentralizing understanding of artistic and humanistic production in cognitive research. Understanding Material-Based Imagination Cognitive Coupling of Animation and User Action in Interactive Digital Artworks